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1¢ INTRODUCTION 
1.1 Phases | to Ill of St. Lawrence Action Plan 


The first five-year Canada-Quebec agreement for the protection, conservation and restoration of 
the St. Lawrence ecosystem was signed in 1988. At that time, the St. Lawrence Action Plan 
targeted four specific areas: protection of the environment, which was focused on reducing 
industrial toxic releases, restoration of disturbed or contaminated areas, conservation of 
biodiversity and the state of the environment (1). 


A second agreement, called St. Lawrence Vision 2000 (SLV 2000), was implemented in 1993 
and ended in March 1998. In an effort to ensure continuity, the objectives of the protection and 
biodiversity components were pursued. New components were added, such as human health, 
decision support, which focussed primarily on the acquisition of scientific knowledge, community 
involvement with the introduction of the Priority Intervention Zone (ZIP) program and agricultural 
remediation (2). 


In view of the success of the two previous agreements, the Canada and Quebec governments 
signed Phase III of SLV 2000 on June 8, 1998, with the objective of pursuing the joint efforts 
focussed on the St. Lawrence River (3). The mandate of Phase III is to «promote access to and 
restore previous uses of the St. Lawrence River with a view to achieving sustainable 
development» (3) (4). The three main objectives are: «to protect the health of the St. Lawrence 
ecosystem; to protect human health; and to promote the involvement of riverside communities» 
(3) (4). Under Phase III, a component focussing on navigation was added to the biodiversity, 
community involvement, agriculture, human health and industrial and urban components. The 
number of government partners in the St. Lawrence Action Plan has increased from five under 
Phase | to 13 under Phase Ill, in addition to non-governmental organizations, such as Stratégies 
Saint-Laurent, the ZIP committees and the Advisory Committee. The Canada and Quebec 
governments plan to invest $239 M over five years, distributed amongst over 30 results to be 
achieved (3) (4). The population of Quebec is the key client targeted by the many impacts and 
outcomes of the action plan. 


1.2 Why implement a performance framework for SLV 2000? 


During Phase II of SLV 2000, the authors of two reports, Revue de Saint-Laurent Vision 2000 
(5) and Etude économique du programme SLV 2000: un exemple concret de développement 
durable (6) recommended that a program such as SLV 2000 should include a performance 
framework in order to establish the linkages between the objectives and the results, clients and 
resources used. A performance framework would increase the understanding of the links 
between the mission, objectives, and environmental problems addressed under SLV 2000 and 
would make it possible to measure the short- and long-term impacts on target clients. The 
performance framework should include indicators of the environmental and socioeconomic 
benefits of the action taken. 
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1.3 Purpose and clients of the SLV 2000 performance framework 


The purpose of the performance framework is to provide SLV 2000 with a management and 
decision-making tool for strategic planning and program review. It is designed to bring together 
the various elements of the action plan into a cohesive whole. It is useful for selecting realistic 
targets for the achievement of results, and for evaluating and identifying avenues of 
improvement that promote the desired performance. It should play an important role in the 
review of SLV 2000, which will be carried out jointly by the Canada and Quebec governments 
upon its completion in 2003. It may also be possible to use elements of the performance 
framework in the midterm review of the action plan, i.e., in 2000-2001, to determine whether the 
resources planned and used in each component are sufficient to achieve the results. 


The performance framework is intended for the evaluators and managers at various levels 
involved in SLV 2000. It can be used by the co-chairs of the coordinating committees (Figure 1) 
for planning purposes and for more effectively analyzing the impact of their work on their clients, 
by managers of the Agreement Management Committee (AMC) and by departmental decision- 
makers, for making informed decisions on the St. Lawrence ecosystem. 


Bea 
ntLaurent § Mfanagement Structure of SLV2000 
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Figure 1: Organization chart of the management structure of Phase III 
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1.4 Work team 


A Canada-Quebec team of evaluators was established in the summer of 1998 to develop the 
performance framework. The team consisted of Sylvie Petitclerc, of the Institutional Affairs 
Branch of the Quebec Environment Department, and Francois Boulanger, of the Corporate 
Affairs Branch of Environment Canada. They developed the work method and presided over all 
meetings with the coordinating committees to develop the first blueprint of the performance 
framework. 


The two analysts of the SLV 2000 Coordination Office, Josée De Guise and Marc Crispin, and a 
student trainee, Payse Mailhot, also took part in the various steps involved in developing the 
performance framework. The analysts of the Coordination Office completed the final version of 
the performance framework with the cooperation of the coordinating committees. 


2. THEORETICAL MODEL FOR MEASURING PERFORMANCE AND ITS COMPONENTS 


Results-based performance measurement has become increasingly popular in governments and 
large companies. This interest certainly stems from the spending cuts of the late 1980s and 
taxpayers’ desire to know what has been achieved through government programs with tax 
dollars (7). Managers also want to be able to demonstrate the value of the action taken. 


Experts agree that strategic decisions and communications that are the most effective and have 
the greatest public impact require measurements of not only financial performance but also of 
the benefits of a program (7). The performance framework provides information on activities, 
outcomes and outputs, i.e., elements of internal management. It also provides information on 
the long-term impacts and achievements that benefit the public. This aspect represents an 
important external dimension for departments since it makes it possible to establish clear 
linkages between the results, internal management and mission as well as the roles and 
responsibilities towards the public. 


The components of the performance framework are logic models, key elements, performance 
indicators and measurement strategies. 


2.1 Logic model 


The logic model (Table 1) presents statements describing the logic of the activities implemented 
to achieve the results of SLV 2000. It is the structured representation of the purpose of and 
rationale for the activities of SLV 2000 within the framework of the results of the Agreement. In 
concrete terms, the logic model represents the linkages between the activities, outputs, 
outcomes and clients, which had not been made until now under the St. Lawrence Action Plan. 


The logic model comprises several statements on, among other things, the activities related to 
the results of SLV 2000 and the outputs, which are defined as the goods and services generated 
by the activities in order to achieve intermediate and ultimate outcomes. 
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The intermediate outcomes are related to the activities and are generated during the period of 
the SLV2000 action plan and for a short subsequent period (1 to 5 years). They are largely 
under the control of the coordinating committees. 


The ultimate outcomes are related to the mission and main objectives of the action plan and are 
generated over the longer term, i.e., 5 to 20 years. While the activities of the coordinating 
committees contribute to them, they are also affected by other actors. 


The logic model also illustrates the clients targeted by the results of the Agreement, the partners 
that contribute to the activities, the users of the outputs and the beneficiaries of the results. 


2.2 Key elements and performance indicators 


A certain number of the intermediate and ultimate outcomes set out in the logic model are 
selected by the coordinating committees to be measured in the performance framework. They 
are referred to as the key elements and are associated with one or more qualitative or 
quantitative performance indicators (Table 2). The most relevant indicators are selected using 
an evaluation grid (Table 3). 


2.3 Measurement strategies 


A measurement strategy is developed for each indicator selected. It sets out the unit of 
measure, the target sought for the indicator, the source of information for describing it, the data 
collection method, the updating frequency and the person responsible (Table 4). 


2.4 Complementary management tools 


Management tools designed to complement the performance framework were developed to 
obtain as complete a picture as possible of the status of the results and their impact on the St. 
Lawrence ecosystem throughout Phase Ill. An internet based management information system 
was implemented to track the status of the thirty-five Agreement results and the budgets 
allocated to them. A joint work team is working on developing and implementing a monitoring 
network to follow the state of the St. Lawrence ecosystem in order to determine whether there 
has been any improvement. Ecosystem monitoring is based on the updating of environmental 
indicators which reflect the condition of the St. Lawrence (8). 


It is important to specify that these three management tools, i.e., the performance framework, 
the management information system and the state-of-the-ecosystem monitoring network are 
dynamic and evolving tools that can be modified and improved to address new needs. 
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Table 2: Excerpt of the key elements and performance indicators from the 
Industrial and Urban component 


Indicators / 
selected for the performance framework (boldface 


. Number of projects carried out 
2. Rate of toxics reduction per project and type 
of activity 
. Time required per project 
. Total cost 
. Level of achievement of environmental 
results other than the reduction of the 18 
toxic substances (EMS, technical) per project 
and type of activity 
6. Level of compliance of the company with the 
authorization certificates issued 
7. Number of firms with an environmental 
management system (EMS) 
8. Level of success in implementing the specific 
elements of an EMS 
9. Changes in the quantity of toxic inputs for a 
project 
10.Rates of reduction of effluent releases 


Key Elements 


INDUSTRIAL AND URBAN 

. Demonstrate that the pollution prevention 
approach is effective in reducing toxics 
(implement pollution prevention projects at 60 
plants) 


. Improve the environmental performance of the 
targeted SMEs 
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Table 3: Evaluation Grid for the Selection of Indicators 


Indicator 
1 = strongly disagree 
2 = disagree 

3 = no opinion 

4 = agree 


Indicate, on a scale of 1 to 5, whether you agree 
with the following questions on the 
selected indicators. 
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2.3 4 5 
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1._Is it related to the results? 
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2. Is it representative and useful to users? ieee. «45 
3. Is it related to the activities? te tee 4 5 


Comparable 


1. Can comparisons over time be made with other 
organizations, activities or standards? 
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Does it effectively represent what is being measured Ue Ces 


unbiased)? 


3. Are the data of high quality (free of error)? 


1. Is it financially feasible? late 455 
: in ti ? 2 4 5 


4) 
2. Is it possible to obtain timely data? 


This indicator will be useful to the managers of the Ci. i Sa 
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3. FROM THEORY TO PRACTICE: WORKSHOPS 


The products of the performance framework were developed through meetings attended by 
over seventy individuals (Appendix |) who were selected for their knowledge of the operational 
activities and broad view of the purpose of these activities in their component and in the action 
plan. It has been shown that the choice of participants is critical to the success of the exercise 
(7). Preparatory meetings for each of the coordinating committees were held between October 
1998 and January 1999. The purpose of these meetings, which brought together evaluators, 
an analyst from the Coordination Office and the co-chairs of the coordinating committee, was to 
provide information on the approach to be adopted to develop the performance framework and 
to ask the co-chairs to identify the workshop participants. 


Given the large number of results targeted by certain components, such as the Biodiversity, 
Community Involvement and Industrial and Urban components, they were divided into 
subcomponents for the purposes of the workshops. A total of 14 workshops, each generally 
two days long, were held between December 1998 and March 1999 (Appendix Il). These 
intensive working sessions comprised five steps: development of the logic model; 
determination of key elements (measured intermediate and ultimate outcomes and impacts); 
identification of the associated performance indicators; selection of the most relevant indicators 
using an evaluation grid and development of a measurement strategy for the indicators 
selected. 


The products from the workshops were developed through consensus among the participants. 
They were subsequently validated by each of the coordinating committees, which lead to the 
elimination of indicators considered less relevant or difficult to document. 


4. PERFORMANCE FRAMEWORK DATA 


The second volume of the performance framework (9) presents all data on the various products 
for each coordinating committee. This section of the report presents only a compilation of the 
performance indicators that will be measured by the coordinating committees. 


4.1 Performance indicators 


A total of 103 performance indicators were selected by the coordinating committees to 
document the results of Phase Ill. The number of indicators selected by each committee is as 
follows: Community Involvement, 37; Agriculture, 11; Biodiversity, 13; Industrial and Urban, 8; 
Navigation, 12; Health, 14; and Communications, 8. The performance indicators associated 
with the results of Phase III are presented in Table 5. In addition to the 35 results set out in the 
Agreement (3), two were added for institutional communications and are included in Table 5. 
The target of each indicator is specified in the measurement strategies presented in the second 
volume of the performance framework. 
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Table 5: Agreement Results and Performance Indicators 


COMMUNITY INVOLVEMENT 


RESULTS | INDICATORS 


The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 


Help build consensus in 14 riverside . Number of partners per sector involved in the 
communities (ZIP) on local environmental implementation of ERAP projects 
issues under the coordination of Stratégies . Number of partners involved in activities other than 
Saint-Laurent the implementation of ERAP projects 
. Number of hectares of protected, restored, conserved 
or enhanced habitat 
. Number of ERAP projects completed or under 
way in the following categories: protection, 
conservation, restoration and enhancement of 
habitats and biological resources, 
recreation/tourism and protection of human 
health 
. Number of plants planted 
. Number of kilometres of shoreline restored (cleaned 
up, stabilized, renaturalized) 
. Amount of waste collected 
. Number of hours of volunteer work provided by 
local partners to the activities of ZIP committees 
. Resources (financial, human and physical) 
invested by the other local partners to implement 
ERAPs and other activities 
. Number of conferences, consultations and 
information and awareness activities generated by 
the ZIP committees 
. Number of harmonization activities (meetings of 
Stratégies Saint-Laurent and ZIP boards, 
development of management tools, etc. 
Support implementation of 150 community . % of the budget of the Community Interaction 
projects arising from ERAPs and the Program (CIP) allocated to ERAP projects 
communities, giving priority to access to the . Number of proposals submitted to CIP 
St. Lawrence and its use addressing each component of SLV 2000 
. Number of community groups participating in CIP. 
Number of person/days of volunteer work allocated to 
CIP 
. Number of projects per ZIP (geographical aspect) 
. Number of CIP projects in the following 
ies: physical works, awareness, 
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RESULTS INDICATORS 
The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 
acquisition and stewardship, enhancement and 
accessibility, studies (generation of knowledge, 
planning, feasibility) 
. % of funding per type of partner 
. Distribution (%) per type of partner involved in CIP 
Multiplier effect of CIP: investments ($) by type of 
partner; volunteer work in $ and person/days; 
loans, donations and services in $ 
. Number of kilometres of shoreline restored (cleaned 
up, stabilized, renaturalized) 
. Amount of waste collected 
. Number of plants planted 
. Number of accesses to the St. Lawrence (new and 
improved) 
Hectares of restored, conserved or protected habitat 
Provide scientific and technical support for ZIP Number of government partners participating in 
committees SLV 2000 (Diversity) involved with SSL and ZIP 
committees 
. Resources allocated by each government partner 
participating in SLV 2000 
. Level of satisfaction of SSL and ZIP committees as a 
function of the offer made by government partners 
. Client satisfaction, i.e., SSL and ZIP committees, 
concerning the transfer of information and 
expertise by the government partners 
participating in SLV 2000 
. % Of ERAP projects which are technically eligible for 
funding under CIP 
. Total number of ERAP projects receiving funding 
under CIP 
. Total number of ERAP projects that have been 
implemented and that have received scientific and/or 
technical support 
Disseminate the best and the latest information . Number of people active in the Biosphere’s 
on the St. Lawrence via the Biosphere’s Ecowatch Network in the St. Lawrence basin 
EcoWatch Network a centre of expertise and (participants) 
scientific popularization on the St. Lawrence . Number of requests to participate in the Biosphere’s 
environment Ecowatch Network in the St. Lawrence basin 
(participants) 
. Number of projects per theme 
. Number of hours spent on the Biosphere’s 
Ecowatch Network projects by the participants, 
including volunteer work 
37. % retention of active members 
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AGRICULTURE 


RESULTS INDICATORS 


The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 


Reduce by 50% the use of pesticides and . Amount of pesticides in kg of active ingredients 
obtain 70% of the area under integrated control per hectare used in targeted crops 
measures by 2003 and follow-up to verify the . Areain hectares under integrated control 
achievement of results measures relative to the total area of targeted 
crops 
Frequency of exceedance of criteria for the 
protection of aquatic life (for surface water) 
Frequency of pesticide detection (in groundwater) 
. Frequency of exceedance of drinking water criteria 
Have 175 agricultural producers in the Boyer . Number of producers who are members of agro- 
River basin join agro-environmental clubs in environmental clubs for the Boyer River 
order to re-establish smelt (modified) . Number of watertight storage structures (livestock 
manure) 
Number of farms with agro-environmental fertilization 
plans 
Frequency of exceedance of the criteria for the 


Validate an indicator on the risks of surface 

water contamination by phosphorus 

Carry out control and inspection in the area of | 10. Number of farms in compliance with the provisions 
agricultural cleaning up and make agricultural respecting watertight storage of livestock manure 
industries conform to the Reglement sur la 11. Number of inspection visits 

réduction de la pollution d’origine agricole dans 

des tributaires du Saint-Laurent 
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BIODIVERSITY 


RESULTS 


Contribute to the safeguard of 35 threatened 
plant and animal species 


Protect 120,000 hectares of natural habitats, 
including 1,660 hectares to be acquired 


Develop and implement 7 management and 
conservation plans for sensitive habitats 


INDICATORS 
The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 


. Number of species whose populations have 


stabilized or are on the rise 
Number of species whose occupied sites have 
been maintained or increased 


. Number of new species recovery plans initiated 


or completed 


. Number of species designated by the Quebec 


government 


. Number of hectares protected 
. Area with protection status (included in the 


120,000 hectares) 


. Status of habitat management plans that have 


been implemented 


Educate the public and raise their awareness of 
the ecological value of the St. Lawrence 


Promote public access to the St. Lawrence by 
developing structures and sites with ecological 
potential 


Assess impacts of water-level fluctuations due 
to climate change and streamflow regulation on 
the St. Lawrence ecosystem and its uses 
Provide forecasts and analyses of the state of 
the St. Lawrence by implementing an integrated 
monitoring system 


8. 
a: 


% progress on the Lachine Canal project 
Number of accesses to the St. Lawrence River 
created or restored 


10. Extent and variety of use of water level 


management models and tools 


11. Level of implementation of the reference 


framework (operationalization of existing 
indicators and evaluation of the feasibility of the 
others) 


12. Level of commitment of partners to the St. 


Lawrence monitoring network 


13. Implementation of a permanent process to 


ensure the sustainability of a St. Lawrence 
monitoring system 
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INDUSTRIAL AND URBAN 


RESULTS 


Find and process information on three industrial 
sectors (metallurgy, metal and chemical) in 
order to prioritize action on 18 priority toxic 
substances (modified 

Develop environmental management tools for 
the metallurgy, metal and chemical industries in 
order to prioritize action on the 18 priority toxic 
substances (modified 

Introduce pollution prevention projects into 60 
plants (20 per sector) 


Provide technical expertise and financial 
support to promote the development, adaptation 
and commercialization of new pollution 
prevention technology or equipment 


Evaluate the economic and environmental gains 
of the pollution prevention projects 


INDICATORS 


The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 


. Rate of toxics reduction per project and type of 


activity 


. Number of pollution prevention projects carried out 
. Level of achievement of the environmental results 


other than the reduction of the 18 toxic 
substances (EMS, technical) per project and type 
of activity 


Proceed to the control and inspection in the 
industrial sector 


Measure toxicity of effluents from three major 
metropolitan areas (MUC, QUC, OUC) and nine 
other municipalities in order to support 
corrective measures 


4. Achievement of all results for measuring the 
toxicity of targeted municipal effluent 
5. Achievement of high quality results involving 
results that passed quality control tests 
characterization carried out in a timely manner 
(taking account of influencing factors) 
normal operation of treatment plants 
6. Number of the 14 industrial plants that completed the 
program 
7. CHIMIOTOX for each industrial plant targeted 
under Phase Il in 1 


Complete the reduction of toxic liquid effluents 
from 14 Phase II priority plants (90% or optimal 
reduction to achieve virtual elimination) - 
modified 

Introduce an environmental awards program for |8. Number of certificates issued compared to the 


the industrial plants targeted under Phases | number of certificates that can be issued 


and Il 
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SHIPPING AND BOATING 


RESULTS INDICATORS 
The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 
Establish a mechanism for monitoring dredging |1. Implementation of environmental monitoring 


activities on the St. Lawrence studies for dredging projects 


Develop tools and guides to improve . Stakeholders’ level of knowledge on the government 
management of environmental risks and threats decision-making process regarding the management 
as part of an effort to ensure sustainable of dredging and sediments 

shipping on the St. Lawrence 


Protect the banks of the St. Lawrence against . Implementation of studies on the impacts of 
the effects of erosion due to commercial navigation on shoreline erosion 
shipping and pleasure boating . Behavioural changes vis-a-vis the speed of 
commercial ships 
. Changes in the practices of recreational boaters 


Introduce regulations or a code of good practice 
governing discharge of ballast water 


Develop and implement a strategy to ensure . Level of involvement per type of government and non- 
sustainable boating and shipping on the St. governmental representative in the development of 
Lawrence in cooperation with the shipping the sustainable shipping strategy 
industry, environmental interests, governments . Level of funding committed by partners to the 
and the public sustainable shipping strategy 
. Adoption of the sustainable shipping strategy 
tabled by the Shipping and Boating coordinating 
committee 
. Level of participation in reflection and public debate 
on the use by each type of representative for each 
geographical area 
10.Quality of the debate on the uses of the St. Lawrence 
River and projects related to navigation 
11.Number and frequency of information transfers to the 
target clients of the Shipping and Boating component 
Implement a management plan for 12.Scientific consensus and level of information of 
contaminated sites that may present a risk for industry, the public and local communities on the 
the ecosystem application of the criteria on sediment quality 


Produce or update tools to improve sediment 
management practices 
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HEALTH 


RESULTS INDICATORS 


The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 


Reduce public exposure to recreational waters . Number of beaches used by the public at which 
with micro-biological hazards water quality is evaluated 
% of municipalities that monitor water quality at 
public beaches in their territory 
% Of municipalities who post adequate notices at 
beaches in their territory 
Proportion of swimmers frequenting clean beaches, 
that have adequate posting and are suitable for 
swimming 
Proportion of the riverside population that swims in 
the St. Lawrence River 
Swimmers attitude and knowledge of health 
issues 


Reduce public exposure to drinking where there |7. Number of new priority chemical contaminants in 
is a risk of chemical and microbiological drinking water subject to characterization to assess 
contamination potential health risks 
Consumer attitudes regarding tap water from the 
St. Lawrence River 


Reduce public exposure to consumption of . % of pregnant women whose level of PCB 
aquatic products where there is a risk of organic contamination is below the safety threshold in 
and inorganic contamination high-risk areas 
. Heath care professionals’ attitude towards and 
knowledge of health issues related to the 
consumption of aquatic products 
. Proportion of riparian land owners engaging in sport 
fishing in the St. Lawrence River 
. Proportion of fishermen who consume catches from 
the St. Lawrence River 
. Consumers’ attitude towards and knowledge of 
health issues related to the consumption of 
aquatic products 
. Number of aquatic product consumption 
practices subject to human health risk 
characterization 
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COMMUNICATIONS 


RESULTS INDICATORS 
The text in boldface corresponds to the indicators selected specifically 
for the Agreement Management Committee (AMC) 
Inform the public, media and target clients of . Number of times SLV 2000 is associated with the 
the activities, programs, outcomes and two governments (Canada, Quebec) compared to 
achievements of the St. Lawrence Action Plan the total number of times it is mentioned 
Vision 2000 . Number of times departmental partners are 
mentioned in articles generated by the 
Promote the harmonization of St. Lawrence communications activities of SLV 2000 compared to 
Vision 2000 communications the number of times they could be mentioned in all 
articles generated 
. Number of references to the targeted strategic 
messages compared to the number of references 
in all articles generated by communications 
activities 
. Number of communications products per 
coordinating committee posted on the Internet 
compared to the total number of products 
developed and included in their communication 
plan 
. Number of articles published in «Le Fleuve» per 
area of activity 
. Public and community perception of all SLV 2000 
activities 
. Level of public knowledge of the state of the 
St. Lawrence River 
. Uses and changes in perception of the St. 
Lawrence River 
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Eight of the 37 results do not have performance indicators. In some cases, coordinating 
committees considered some results to be less important in evaluating their component's 
performance. In other cases, the updating of indicators for these results would have been too 
complex or would have required a level of funding that is not currently available. It is 
anticipated that the coordinating committees should develop, by the end of the action plan, a 
qualitative measure or index for evaluating these results which do not have a performance 
indicator. In addition, managers have basic information on the status of all results in the 
management information system. 


At the request of the Agreement Management Committee (AMC), 48 performance indicators 
were selected by the coordinating committees as being the most representative and most 
relevant to their component (see Table 6). These indicators should be more closely monitored 
by the AMC and could be used for various management purposes. 


4.2 Categories of performance indicators 


In order to gain a better representation of the various aspects of SLV 2000 covered by the 
performance indicators, they were divided into three categories: environmental indicators, 
indicators of intermediate outcomes and indicators of products and services (Table 7). 


The environmental indicators measure the outcomes associated with the mission of SLV 2000, 
i.e., the progress made towards improving the state of the St. Lawrence ecosystem, human 
health and community involvement. They contribute to the measurement of the long-term 
outcomes (10-20 years). The environmental indicators of the performance framework are 
complemented by those used to monitor the state of the St. Lawrence ecosystem, which are 
developed and implemented by a team established specifically for this purpose. 


The indicators of intermediate results measure the progress made in respect of the Agreement 
results and have a five-year horizon. The indicators are complemented, in respect of the 
budget associated with each result, by the information contained in the management 
information system. 


The indicators of products and services cover the administrative and management aspects, 
such as the number of reports published or the number of projects or activities implemented in 
respect of the Agreement results. The latter, which are more specifically management 
indicators, are short term indicators, often even annual. 


The use of the grid presented in Table 7 enables us to see that very few environmental 
indicators were retained by the coordinating committees. With respect to the intermediate 
indicators, i.e., those that are directly related to the achievement of the Agreement results, we 
see that only about 20 will be measured. Most of the indicators of the performance framework 
are in the products and services category. This likely reflects the fact that the government 
partners have more experience in identifying these management elements, which are related to 
the departments’ annual planning exercises. 
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Table 6: Selection of indicators of the performance framework for the 


13. 
14. 


hey 
16. 
ive 
18. 
19; 
20. 
at. 
ei 
23. 
24. 


Agreement Management Committee (AMC) corresponding to the text 
in boldface in Table 5 


Community Involvement 
Number of partners per sector involved in the implementation of ERAP projects 


Number of ERAP projects completed or under way in the following categories: protection, 
conservation, restoration and enhancement of habitats and biological resources, recreation/tourism 
and protection of human health 


Number of hours of volunteer work provided by local partners to the activities of ZIP committees 


Resources (financial, human and physical) invested by the other local partners to implement ERAPs 
and other activities 


% of the budget of the Community Interaction Program (CIP) allocated to ERAP projects 
Number of proposals submitted to CIP addressing each component of SLV 2000 


Number of CIP projects in the following categories: physical works, awareness, acquisition and 
stewardship, enhancement and accessibility, studies (generation of knowledge, planning, feasibility) 


Multiplier effect of CIP: investments ($) by type of partner; volunteer work in $ and person/days; 
loans, donations and services in $ 


Resources allocated by each government partner participating in SLV 2000 


Client satisfaction, i.e., SSL and ZIP committees, with the transfer of information and expertise by the 
government partners participating in SLV 2000 


Number of people active in the Biosphere’s Ecowatch Network in the St. Lawrence basin 
(participants) 


Number of hours spent on the Biosphere’s Ecowatch Network projects by the participants, including 
volunteer work 


Agriculture 
Amount of pesticides in kg of active ingredients per hectare used in targeted crops 


Area in hectares under integrated control measures relative to the total area of targeted crops 


Biodiversity 
Number of species whose populations have stabilized or are on the rise 
Number of species whose occupied sites have been maintained or increased 
Number of new species recovery plans initiated or completed 
Number of species designated by the Quebec government 
Number of hectares protected 
Area with protection status (included in the 120,000 hectares) 
Status of habitat management plans that have been implemented 
% progress on the Lachine Canal project 
Extent and variety of use of water level management models and tools 


Level of implementation of the reference framework (operationalization of existing indicators and 
evaluation of the feasibility of the others) 
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eo: 
26. 


Zi. 
28. 


ash 
30. 
ail 


32. 
33. 
34. 
35. 
36. 


37. 
38. 
30. 


40. 


41. 


42. 


Level of commitment of partners to the St. Lawrence monitoring network 


Implementation of a permanent process to ensure the sustainability of a St. Lawrence monitoring 
system 


Industrial and Urban 
Rate of toxics reduction per project and type of activity 


Level of achievement of the environmental results other than the reduction of the 18 toxic 
substances (EMS, technical) per project and type of activity 


Achievement of all results for measuring the toxicity of targeted municipal effluent 
CHIMIOTOxX for each industrial plant targeted under Phase II in group 1 


Number of certificates issued compared to the number of certificates that can be issued 


Shipping and Boating 
Implementation of environmental monitoring studies for dredging projects 
Implementation of studies on the impacts of navigation on shoreline erosion 
Behavioural changes vis-a-vis the speed of commercial ships (shoreline erosion) 
Changes in the practices of recreational boaters (shoreline erosion) 


Adoption of the sustainable shipping strategy tabled by the Shipping and Boating coordinating 
committee 


Health 
Swimmers attitude and knowledge of health issues 
Consumer attitudes regarding tap water from the St. Lawrence River 


% of pregnant women whose level of PCB contamination is below the safety threshold in high-risk 
areas 


Consumers’ attitude towards and knowledge of health issues related to the consumption of aquatic 
products 


Number of aquatic product consumption practices subject to human health risk characterization 


Communications 


Number of times SLV 2000 is associated with the two governments (Canada, Quebec) compared to 
the total number of times it is mentioned 


Number of references to the targeted strategic messages compared to the number of references in 
all articles generated by communications activities 


Number of communications products per coordinating committee posted on the Internet compared to 
the total number of products developed and included in their communication plan 


Number of articles published in «Le Fleuve» per area of activity 
Public and community perception of all SLV 2000 activities 
Level of public knowledge of the state of the St. Lawrence River 


; ‘Uses and changes in perception of the St. Lawrence River 
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Performance Framework - SLV 2000 Phase II 


Di IMPLEMENTATION OF THE PERFORMANCE FRAMEWORK 
5.1 Collection of information pertaining to performance indicators 


The performance framework is implemented through the collection of information pertaining to 
performance indicators. The coordinating committees are responsible for updating their 
performance indicators in accordance with the frequency determined at the workshops and as 
specified in the measurement strategies. Appendix Ill presents the indicators by component 
with their updating frequency. 


As a next step, the measurement strategies will be integrated into the internet based 
management information system of Phase III. The management information system will thus 
become a single-window tool that will be used by the designated representative(s) of the 
coordinating committees to document their performance indicators, budgets and status of 
results. 


5.2 Logic model database 


The information in the logic models was put into a local database available at the SLV 2000 
Coordination Office. The database makes it possible to dynamically link the various elements 
of the logic model and to see, for example, how a change in an activity can affect the 
components. The current version of the database will not be integrated into the management 
information system, even though it is compatible with it. For now, the logic models are available 
as tables in the management information system (management tools section). 


6. PHASE IIl EVALUATION ISSUES 


The performance framework is an important tool for conducting the program review. For 
Phase Ill, a mid-term review is planned for 2000-2001 and an review of the action plan is 
scheduled for the final year, i.e., 2002-2003. These two exercises will be conducted jointly by 
the Canada and Quebec governments. 


The purpose of the mid-term review is to determine what progress was been made towards 
achieving the planned results and to decide on the directions to take or on the need to 
reallocate resources to ensure optimal success of SLV 2000. It is clear that with respect to 
some results for which progress can only be seen in the long term, the mid-term review will 
occur at too early a stage to evaluate trends in respect of the achievement of targets. However, 
for shorter-term results, it will be possible to obtain relevant overviews that can be used to more 
effectively focus or verify the effectiveness of certain aspects of SLV 2000. 


The performance framework is designed specifically for the review of Phase Ill. It will allow 
managers to conduct the preliminary analysis required for this exercise. During the review, the 
evaluators will ask managers questions on three specific aspects of Phase Ill, i.e., relevance, 


effectiveness and efficiency (5) (7). Apart from the performance framework, the evaluators will 
also use data from the management information system, interviews with the people actively 
involved in SLV 2000 and various reports. 
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The relevance of SLV 2000 relates to its justification in terms of its concept, development, 
implementation and partnerships (5) (7). Are the results to be achieved true priorities for the 
strategic objectives of «protection of the health of the ecosystem, protection of human health 
and involvement of riverside communities to improve access and to restore uses of the 
St. Lawrence River» (3) (4)? How do the activities implemented contribute to achieving the 


results? 


The effectiveness of SLV 2000 relates to the achievement of the results, the impacts on the 
St. Lawrence ecosystem and partnerships. The principal questions asked are: Were the 
targeted results achieved? Was progress made towards the main objectives of SLV 2000? Are 
partners really involved in the achievement of the results? Is SLV 2000 contributing to 
improving the St. Lawrence River? The development of new skills and expertise and learning 
from the mistakes made in Phases | and II are also reflections of effectiveness. The 
satisfaction of the clients targeted by the outputs of Phase Ill, including those related to 
communication and dissemination of scientific knowledge on the St. Lawrence, is an integral 
part of the evaluation of the effectiveness of SLV 2000. The integration of the information 
obtained on the ecosystem as well as the use of the SLV 2000 outputs in decision-making 
concerning the St. Lawrence ecosystem are clear evidence of the effectiveness of the program 


(5) (7). 


The efficiency of SLV 2000 relates primarily to the costs of achieving the results. Are the costs 
considered reasonable? The question of efficiency also relates to the choices made by 
managers when allocating resources, to different results or components. Are the key priorities 
allocated sufficient resources relative to the less important aspects of the action plan (5) (7)? 


In the review that will be carried out at the end of Phase Ill, the managers will be asked difficult 
questions. However, the challenge is worthwhile. The conclusions and recommendations of 
the review will be useful in improving the initiatives to be taken regarding the St. Lawrence as 
well as with respect to partnerships. This exercise is critical for the evaluation of the impacts 
and outcomes of the activities of SLV 2000 on the St. Lawrence and, if required, to plan of the 
next Phase. 


Th: RECOMMENDATIONS 


The development of the performance framework is first and foremost a job for experts. In 
developing a product as far-reaching in scope as the performance framework, it is critical to 
ensure the availability of human resources throughout the development process. Because both 
evaluators left in April 1999, the version of the performance framework presented to the 
Coordination Office was incomplete in several respects and had not been validated by all the 
coordinating committees. This resulted in a delay of several months in the development of the 
performance framework. 


tan Indicators 


Because it must reflect changes in planning that may arise during the course of the action plan, 
the performance framework of Phase Ill is a dynamic, evolving tool. At this point, the 
performance framework is not perfect. The performance indicators do not cover all aspects 
covered by SLV 2000: some results have no indicators, while others have no indicators of 
intermediate outcomes (progress towards targets), which would have to be developed. There 
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are few environmental indicators (longer term). Linkages should be established with the team 
working on the state of St. Lawrence ecosystem monitoring, who will follow trends in 19 
environmental indicators. In addition, general indicators for Phase III have yet to be developed. 


The level of indicators put forward by the coordinating committees is unequal. It is important 
that the coordinating committees examine this question and look to see how they can document 
the impact of their work on the achievement of the Agreement results, on the ecosystem, on 
human health and on community involvement in the absence of indicators in the three 
categories presented in Table 7. 


7.2 Recommendations of the Advisory Committee 


The Advisory Committee made a series of recommendations to AMC in its working paper on 
the performance framework (10), many of which were taken into account in preparing this 
document. The committee recommended, among other things, the inclusion of general 
indicators for Phase Ill for evaluating integration, coordination, cooperation and management. 
It also believes that the communication and dissemination of scientific information on the St. 
Lawrence ecosystem to decision-makers and the public should be a major priority in Phase III. 
This element will not be measured by the Communications coordinating committee or by any of 
the other coordinating committees with performance indicators, but is an integral part of the 
evaluation of the effectiveness of SLV 2000, as specified in section 6. The satisfaction of the 
various target clients of the SLV 2000 action plan, namely the public, the ZIP committees and 
all decision-makers involved in the St. Lawrence, must be evaluated, despite the fact that no 
indicator currently appears in the performance framework. 


7.3.  %Implementation 


The coordinating committees must continue collecting information for updating the performance 
indicators at the established frequency. The information from the performance framework must 
be used as often as possible in communications on the action plan, whether it be simply for 
presenting work under way or for planning purposes, the midyear review or the year-end 
review. The performance framework should be used regularly in meetings of SLV 2000 
managers to discuss the impact of the work being carried out under Phase Ill. 


8. CONCLUSION 


The performance framework for Phase III is a major, innovative effort, as reflected by the 
number of people who took part in the workshops (Appendix |!) and the volume of information 
flowing from them, which will be presented in Volume II (9). It is the first exercise of its kind to 
be carried out since the first Canada - Quebec action plan on the St. Lawrence in 1988. It 
makes it possible to establish linkages between the mission and three main objectives of Phase 
lll and its results, activities, outputs, clients and outcomes. The performance framework was 
developed jointly with the participation of all the coordinating committees, which showed 
considerable interest in this new management tool. 


The performance framework is a major decision-making tool for planning and the final review of 
Phase Ill and, eventually, for the strategic planning of the results of Phase IV, if it’s applicable. 
It informs each coordinating committee of its situation relative to required performance and of 
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the aspects on which it will be evaluated. However, the analysis will be backed up by additional 
information from other sources, such as the management information system, interviews with 
the stakeholders of Phase III and published reports in order to provide a clearer picture of the 
performance of government with respect to the St. Lawrence ecosystem. 


The thinking required to develop the performance framework, i.e., to establish linkages between 
the various elements of SLV 2000, has been a very fruitful exercise for everyone involved. It 
has enabled the coordinating committees and managers to examine and summarize the work 
done in Phase III and to communicate their work from the perspective of the needs of and 
usefulness to clients as well as its environmental benefits. The same type of thinking should be 
maintained for any new activity added by the planning process and must also be very apparent 
in SLV2000 communications. 
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Appendix!: List of participants for the fourteen workshops to develop the performance 


framework 


Community 
Involvement 


Joint community action 
Marie-Josée Auclair 
Yolaine St-Jacques 
Claire Lachance 

Thanh Thao Pham 

Marc Hudon 

Yves Lefebvre 

Guy Larochelle 
Jean-Francois Bibeault 
Claire Laliberté 


Scientific and technical 
support 

Marie-Josée Auclair 
Yolaine St-Jacques 
Thanh Thao Pham 
Jean-Francois Bibeault 


Support for community 
action 

Yves Lefebvre 

Daniel Robitaille 


Dissemination of 
scientific knowledge 
Michel Provencher 
Marie-Josée Auclair 
Yolaine St-Jacques 
Thérese Baribeau 
Michelle Sincennes 
Jean-Francois Bibeault 


Agriculture 


Denis Sanfagon 


Raymond-Marie Duchesne 


Alain Verreault 
Marcel Gaucher 


Environment Canada 

Environment Canada 

ZIP Haut Saint-Laurent 

Environment Canada 

Stratégies Saint-Laurent 

Ministere de l’Environnement Québec 
Stratégies Saint-Laurent 
Environment Canada 

Centre de santé publique de Québec 


Environment Canada 
Environment Canada 
Environment Canada 
Environment Canada 


Ministere de l'Environnement Québec 
Environment Canada 


Environment Canada 
Environment Canada 
Environment Canada 
Environment Canada 
Environment Canada 
Environment Canada 


Ministere de l’Agriculture, des Pécheries et de |’Alimentation du Québec 
Ministere de l’Agriculture, des Pécheries et de I’Alimentation du Québec 
Ministere de l’Environnement Québec 
Ministere de l’Environnement Québec 
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Biodiversity 


Habitats 

Luc Foisy 

Gleason Thibeault 
Raymond Cournoyer 
Yvon Mercier 
Normand Traversy 


Management of Water 
Levels 

Christiane Hudon 
Richard Laurence 
Gilles Harvey 

Marc Mingelbier 


Species 

Jacques Prescott 
Gildo Lavoie 
Michel Huot 
Raymond Lemieux 
René Lesage 


Ecosystem Monitoring 
Yves Grimard 

Guy Demers 

Thanh Thao Pham 


Industrial / Urban 


Industrial 

Jacynthe D’amours 
Claire Marier 

Francine Perron 
Claudette Bégin 

Alain Bernier 

Réjean de Ladurantaye 
Jean Tremblay 
Anne-Marie Coutu 
Fadila Matejek 


Urban 

Jacynthe D’amours 
Alain Bernier 

Marc Villeneuve 
Richard Dalcourt 
Robert J. Tétreault 


Canadian Heritage 

Ministére de |’Environnement du Québec 
Société de la Faune et des Parcs du Québec 
Environment Canada 

Société de la Faune et des Parcs du Québec 


Environment Canada 
Environment Canada 
Société de la Faune et des Parcs du Québec 
Société de la Faune et des Parcs du Québec 


Ministére de l’Environnement du Québec 
Ministére de l’Environnement du Québec 
Société de la Faune et des Parcs du Québec 
Environment Canada 

Société de la Faune et des Parcs du Québec 


Ministére de l'Environnement du Québec 
Ministére de l’Environnement du Québec 
Environment Canada 


Ministére de l’Environnement du Québec 
Environment Canada 
Environment Canada 
Ministére de l’Environnement du Québec 
Environment Canada 
Environment Canada 
Environment Canada 
Environment Canada 
Ministére de l’Environnement du Québec 


Ministére de l’Environnement du Québec 
Environment Canada 
Environment Canada 
Environment Canada 
Ministére de l’Environnement du Québec 
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Shipping and Boating 


Pierre Rouleau 
Gervais Bouchard 
Laurence Mazaudier 
Serge Gonthier 
Jér6éme Faivre 

Marc Hudon 
Francine Richard 
Vincent Jarry 


Health 


Michele Bélanger 
Richard Larue 
Jacques Grondin 
Dave Berryman 
Claire Laliberté 


Communications 


Raymonde Goupil 
Suzanne Bourget 
Marc Hudon 
Nathalie Howson 
Clément Dugas 


Agreement 
Administration 
Marc Crispin 
Josée De Guise 
Francois Duchesneau 


Fisheries and Oceans Canada 

Fisheries and Oceans Canada 

Fisheries and Oceans Canada 

Ministére de Environnement du Québec 
Ministére des Transports du Québec 
Stratégies Saint-Laurent 

Fisheries and Oceans Canada 
Environment Canada 


Ministére de la Santé et des Services sociaux 
Health Canada 

Centre de santé publique de Québec 
Ministére de l’Environnement du Québec 
Centre de santé publique de Québec 


Ministére de l’Environnement du Québec 
Environment Canada 

Stratégies Saint-Laurent 

Health Canada 

Environment Canada 


Environment Canada 
Environment Canada 
Société de la Faune et des Parcs du Québec 
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Appendix Il : Dates when the fourteen workshops to develop the performance 
framework were held 


Committees =| +~—_— Meeting with co-chairs of | Workshop 
ee coordinating committees — 


Community October, 19 1998 
Involvement 


e Joint community December, 1-2 1998 
action January, 14 1999 


Support for December, 15 1998 
community action 


Scientific and January, 13 1999 
technical support 


Dissemination of January, 13 1999 
scientific knowledge 


Biodiversity January 11 and 26, 1999 
Habitats March, 8-9 1999 
Species February, 15-16 1999 


Management of March, 17-18 1999 
Water Levels 


Ecosystem March, 29 1999 
Monitoring 


Industrial et Urban January 26, 1999 
e Industrial March, 25-26 1999 


Urban March, 12 1999 


December 21, 1998 February, 23-24 1999 
December 11, 1998 January, 6- 7- 27 1999 


January 19, 1999 February, 18-19 1999 


35 


9€ 


dID 0} payedojye 410M J99]UNIOA JO SAeP/uOSJed JO JeqUUINN *G} 
diO ul Burnediomed sdnoib Ayunuiwod jo sequiny “vt 

0002 A1S 
JO JuQUOdWIOD YoRa Bulssaippe jd 0} peyiwiqns sjesodod jo JequNN “€} 


sysloid dVH3 
0} payeooy|e (dqiD) wesbosg uoNoesayuy AyjuNWIWOD ey} Jo yeHpng ay} Jo % ‘Zt 


NOILOV ALINNIWWOD ¥O4S LYOddNS 
"0}8 ‘S]OO} JUeBUUaDeUBW JO JUBLUAOJaA~Ep ‘spieog d|Z 
pue juaine-jules saibaye1S Jo shunoow) SaiIANoOe UONeZIUOWUeY JO JOQUINN “| } 
S89IWWOD g|Z 9u} Aq payesauaD sai}IAOe 
SSOUBJEME PUL UONEWJOJU! PUB SUOI}E}|NSUOD ‘SBOUSJBJUOD JO JOQUWINN ‘OL 
SOIIANO JOUJO pue SGYHy JUsWAI|duU! 0} SIgued 
je00| JaUJO au) Aq paysenu! (jeo1IsAyd pue UeWNY ‘}elOUeUI}) SPDINOSeY 6 
SO9}ILUWOD IZ JO 
SOIIANOe BU} O} SIQUPEd JeD0] Aq PapIAOId YOM JB9JUNJOA JO SINOY JO JOQUINN “8 
p9}0a||09 a}SBM JO JUNnOWY “2 
(pezijeunyeusl 
‘pazijiqeys ‘dn pauea|d) pas0jSed SUlJOJOYS JO SOdJSLWO|Iy JO JBQUINN “9 
pejuejd sjuejd jo JOqUINN “S 
yyyeay UeWWNY JO UONOe}O1d pue WSHNO}/UO!}ees09/ ‘Seosnosa |eo|HojoOIq 
pue syeyiqey JO JUBWBOUBYUS PUL UONEJO}SOI ‘UOIJEAIOSUOD ‘UO!}09}01d 
-seu06eye0 Buimojjo} au} ul ABM JapunN JO payejdwoo sjoeloid qYHF JO JBQUNN ‘Py 
JEWIGeU PSOURYUS JO PAAJBSUOD ‘pas0}SaJ ‘pe}0e}O/d JO Sale}OaY JO JOQUNN “€ 
syoaloid dVH3 
JO UO!}EJUSWA|diU! BY} UY} JAUJO SOAS Ul PSAJOAU! SJOUPE JO JAqUINN “Z 
sjoaloud 
dVHq JO uonejuaWejdu! au} UI PaAJOAU! JOJOeS Jed SJQUUE JO JEqUINN 


NOILOV ALINNWIWOD LNIOP 


6 


| £002-2002| 200z-L00z| 100z-0002| 0002-6661 | 6661-866 | 
[See EE ee ee 


LNAWSATOANI ALINNINWOD 


SHOLVOIGNI 


$10]B9IPUl YOMOWes BOURWOJJed ay} Bulyepdn 40} Aouanbeig : [|| xipueddy 


Ill @SCYUd O00Z ATS - YOMAWeI4 BQUBUIOLad 


Ze 


@WeauU} Jad syoaloud jo JOQqUINN "SE 

(syuedioied) uiseq eousIMe7] “IS 

OU} Ul YIOMION YO}EMOOF" S$,asaydsoig au} ul ayedioed 0} s}senbed Jo JOqUINN ‘ve 
(sjuedioied) uiseq aouesMe7] “1S 

OU} Ul YIOMJBN YOJEMOOF” S$,asaydsoig eu} Ul BAI}Oe ajdoed Jo JeqUINN “CE 


3903 1MONM OISILNSIOS 40 NOILVNIWASSIO 

yoddns jediuydo9} J0/pue OijUSIOS PSAI@d9I 

eAey yeu) pue pe]UeWea|duI! Uaeq aAeY yeU} S}eloId qYHIF JO Jequunu |e}0] *ZE 

did 4epun Burpuny Bulraieoes syoeloid GHA JO Jequunu je}O] “LE 

di Jepun Buipuny 40} ajqiBye Ajpeoiuyoo) ave yorum syoaloid qYHA JO % ‘OE 

0002 AlS 
ul Buryediomed siguped yuawUsaA0b oy) Aq esiedxe pue UO!eUUOJU! 

JO J8JSULI} BU} HulUIBOUOD ‘See}IWWLUOD qjZ PUue TSS *9"! ‘UOHOeJsHeS JUdI|D “62 
siguued juaWUJeA0b Aq apew 

J9}JO BU} JO UOHOUN e Se SEeB}WILUWOD qjZ Puke JSS JO UOHOBIsi}es JO [9A97] “BZ 

000Z AIS u! Bunedioed weuped juswUsJeAOD yore Aq payeooy|e SSdINOSOy "ZZ 

SOSHIWLWOS qi/Z PUB TSS YUM 

0002 AS ul Bunediomed suauyed yuawUJeAOB Jo JaqUNN "92 


LYOddNS IWOINHOSL GNV SISILNAIOS 
yeVIGey pa}0a}01d 10 PAAJOSUOD ‘paJO}SOJ JO SB1E}OSH * 
(peAoidwi pue Mau) BDUdIME] ‘JS BU} 0} SeSS800R JO JOQUINN "7Z 
pejurjd sjuegjd jo JoquNnN ‘€Z 
P9}09}|00 a}SeM JO JUNOWY “ZZ 

(pezijeunyeuss 
‘peziiqeys ‘dn pauea|d) pas0}Se1 SUI|@JOYS JO SBJ}OWO]!y JO JOQUINN "|Z 
$ Ul S8DIABS Puke SUO!}eUOp ‘SueO] -SAep/UOSJEd pue 
Ul YIOM JBEUNJOA “euUped Jo adAy Aq ($) SJUBWSBAU! :qID JO Pao JOIGHINW “OZ 
dID ul perjoaul Jeuyed jo df} Jad (%) UONNqUISIG “61 
Jouued jo adj 19d Buipunj jo % “gf 

(Ayjiqiseey ‘Buiuuejd ‘afpe;mouy jo uolyesoueb) seipnys 

‘Ayiqisseooe pue juaWUaOUeYUA ‘diyspieMes}s puke UOIISINboe ‘sseuaseMe 
‘syiom jeoisAyd :savo6eye0 Buimojjo} au} ul $yOelO1d gID JO JOQUINN *Z1 


PeAOAUI! (AjISIBAIG) 


(148q0}00 
pue 
Aueniqe4) 


Z Z re (yoedse jeoiudeiBoeb) gjZ 48d s}oeloid Jo JOqUINN “OL 
€002-2002| 2002-1002| 1002-0002 | 0002-6661 | 6661-8661 
YVSA SYHOLVOIGNI 


Ill PSCYd 000Z ATS - YOMaWes4 BDUBUIOLAad 


Be 


SHSIA UONOSASU! JO JOQUINN * 
SINUBLW YOO}SAAT| JO BHe10}S 
juBeyem Hunoedsead SUOISIAOId OU} YM BOUe!|GWOD U! SUE} JO JOQWNN “OL 
(4eyeM BOeUNS 
JO}) api] OeNbe Jo UONDS}OId BY} JO} BG} SU} JO BOURBPB9OXO JO Aouenbeli4 ‘6 
Sued UOIeZI|IPO} [e}USLUUOIIAUS-O1He UYIM SUE JO JOQUINN “8 
(ainuewW YOO}SEAI|) Seunjonsys BHesojs yyHpeyeM Jo JequINN “2 
JEAly 4aA0g 
Qu} JO} SGNjo jeJUeLUUOJIAUS-O1He JO SIBQUIAL B12 OYM SJBONPOJ JO JOQUINN *9 


BlJa}119 JeyeEM Buruup Jo eouepsedxe jo AouenbaJ4 
(soyempunoJ6 ul) UOQDa}ep eplonsed jo Aouenbel4 “y 
(49}eM OOeLNS 
jl) oeNbe Jo UOI}Oa}O1d OY) 40} Bl4ajUD JO BOUepeeoxe Jo AOUANbely “€ 
sdoJo pajyebie} 
JO Base [e}0} OU} O} BAI}Eja1 SounseeW jO4]UOD peyesHejul JopuN SeseOeY Ul PAY “2 
sdojo 
payee} ul pasn eeyoey Jad sjuaipas6ul aanoe jo 6y ul sepionsed jo junowY 


BI9}U9 JO BOUePBBOXE YUM AURA |IIM 


[| €002-200z| 002-1002] 1002-0002] 0002-6661 | 6661-8661 


JO} 


) 


3 


SYOLVOIGNI 


SaYNLINOIESDV 


SIOQWIOW BAI}OS JO UO!}US}O1 % “LE 
YIOM JeEJUNJOA Bulpnjou! ‘syuedioied 
au} Aq syoaloid YIOMION YOYEMODF S,as9yudsolg ey} UO jUadS SANOY JO JOQUINN “YE 


| €002-200e| zo0z-L00z| 1002-0002] 0002-6661] 6661-8661 | 


yVSA SYOLVOIGCNI 


Ill @SCYUd 000Z ATS - YOMEWIeIY BQUBULUOPad 


6E 


wia\sks BulO}UOWW SOUSIME] “JS 
@ jo Ajiiqeureysns ay} ainsus 0} sseo0id jUQUeWWEd B JO UO!E}USLUE|dUU] “E | 
YIOMJOU HULOPUOW BOUBIME] "JS OU} O} SIDULEC JO JUSLUFI|LULWOD JO |@AB'] “ct 
(sueyJO aU Jo Ajiqisees ay} JO UOWeN|eAS puke Sioyeo!pu! Bunsixe 
JO UOITEZI|EUOI}EIOdO) YIOMALWEI} 9OUSI9J94 OU} JO UoI}eJUSWI|duUI! JO [SAS] “| 4 


SNIHNOLINOW WALSASOO0A 

S00} pue sjepow juswaebeuew jargj 18}eM JO Asn JO AoA puke jUe}X3 “Ol 
S13A31 YSLVM SAO LNAWSDVNVW 

Pa10}SO1 JO Pa}e9lO JOAIY SOUBIME] “JS OU} 0} S8SSBO0R JO JOQUINN = °6 
yoaloud jeueg aulyoe au} uO sseuBoid % ‘8 

pajuswe|dw! useq evey yeu) Sued juswebeuew yeyqey jo SNES “2 
S0Je}0OY OOO OCI Ou} Ul pepnjou!) SNye}s UOI}OS}OIG YM Bay *9 

pejoejoid sauejoey jo JeqUINN ‘G 

SLVLIGVH 

JUBWIUIJOAOD DEqENH eu; Aq peyeubisap saldeds jo JOQUINN “V7 

paja|duuoo Jo peyenjiul suejd Arenooes Saloeds Mau JO JOQUINN "€ 

pasealoul JO peulejulew Uae SAeY SE}IS Paldnd90 BsoyM SEl9adS JO JOQUINN “¢ 
QS QU} UO aJe JO pazi|iqeys aAey SuoHeindod asoum seloeds Jo JequUINN “} 


€00z-200z| 2002-1002] 1002-0002 0002-6661 | 6661-8661 | 


SHOLVOICNI 


ALISHSAICGOIS 


Ill PBSEYd 000Z ATS - lOMaUIes4 BOUPUOLAd 


Ov 


penssi eq 
UBD JEU} SO}EDIJIUSO JO JOQLUNU OU} O} PaJedLWOD PaNSs! S9}eOI}!99 JO JOQUINN “8 
Ul || BSeud JepUN payebe} JULIA ;eLWsNpu! YOeRS 410} XKOLOIWIHO ~ 
OY} PS}9|GWOD }eU} S}UBIC |el4}SNPU! 7} OU} JO JOQUINN © 
S}ue|d Jud} e9} JO UOI}E18CO |BWJOU 
(siojoe} Hulouenyjul 
jo yunoooe Bulye}) souueWw Ajauly & Ul }NO PaLWed UO!EZUEJOeIeEYO 
$}$9} |04]U09 Ajijenb passed yeu} $}/Nsed 
Buiajoaul s}jnsei Ayyenb ybiy jo yuBWaAaIYyoY * 
juan|jJe 
redioiunw pa}a61e} jo Aj191x0} ay} HuluNseewW JO} S}NSed [|e JO JUBWISAIYOY * 


joeloid yoRa JO Pua Ou} ye ‘A|QeueA 


€002-2002| 2002-1002| 1002-0002} 0002-6661 | 6661-8661 
yYVSA SHOLVOIOCNI 


NV&dn / IVIELSNGNI 


Ill @BSCUd OO0Z ATS - YOMEWILIY BOUBUUOLAd 


Lv 


Say IWWOS Hulyeulpsooo Dulyeog 

pue Buiddiys eu Aq pajqe} A6ayesjs Buiddiys ajqeureysns 34} jo uondopy “Zh 

sjooloid BuiBpaup 40} saipmys BuvoyUOW jeyUSWUOIIAUS 40 uole}USWa|dW] *} | 

SJayeog |euo!}eas091 JO SEOIOeId Buy} Ul SBHULYD © 

sdiys jelou@uW0d Jo peeds au} SIA-e-SIA SeHuBYS JeINOIAeYoq 6 

UOISOJ9 eUljaJOYs UO UOHeHIAeU JO sJOedU| BY} UO SEIPN}s JO uolejuswajdw| “g 

jWenb JUWIPES UO BY9}O ay} JO UOHeD!|dde oy} UO SEa}JUNWILWOS 

je00] pue o1qnd auj ‘ANJSNPUI! JO UOIPEWOJU! JO [BAB] PUB SNSUBSUOD SIUSIOS “2 
uolyebiAeu 0} payejed 

sjoaloid pue JaAIY SOUBIME] ‘JS BU} JO SASN ay} UO B}eGEp 9} JO AyenD “9 
sjuawipes pue Buibpaup jo ywowebeuew ou} Buipsebe 

sse00id BuIyeW-UOISIOep JUSLWIUJAAOH ay} UO SBHpa|MOUyY JO [EAB] SIEP|OYSHeIS “SG 

. juauOdWOO Bulyeog puke Buladiys 

QU} JO S}USI|O JeH1e} OU} O} SIBJSUI} UO!PELUJOJU! JO Aouenbe.j pue JequnN “y 
vale jeoiudeiBoab yore 10} eAnejUsseidai JO 

adfj yore Aq asn au} uo ayegap dI\qnd puke Uo}deJo4 U! uonedioiped jo jeA87] “€ 

ABayens Buiddius ajqeureisns ayy 0} sioued Aq pey!wuwod Bulpuny jo jeAe7] “2 
ABayesjs Buiddius ejqeulejsns au} JO JUSLUdOJSAEP OU} U! aAlyejusseidad 

yeJUBUUJEAOH-UOU PUe JUBWLUAAOBH jo adAj Jed JUBWAAJOAU! JO [BAB] “fh 


SHOLVOIGNI 


ONILVOd GNV ONiddIHS 


Ill @S2Ud 0002 ATS - OMAWEL4 BOUPUIOLAd 


ov 


pea}yuewinoop 
ugeq aAey yeu} SesOdind jeuld|paw 410} pasn syonpoid oenbe jo JOqUINN “9L 
uolezuayOeeYoO 4S 
uyyeay UeWUNY 0} JOelqns seooesd UOI|dUNSUOD yONpoJd oeNbe jo JOQUINN “St 
sjonpoid onenbe jo uoldwuinsuoo 
QU} 0} peyejai Sanssi yyeoy jo eHpajmouy pue SPJeMO} SpNyye SJBWNSUOD ‘Vy | 
JOAIY BOUBIME™] "JS BU} WO1J S8YO}EO BLINSUOD OYM UBWUSYSI} JO uoIwoOdold “4 
JOAIY BOUSIME] 
"1S ey} ul Bulysy ods ul Buibebue siauMo pue| uewedu Jo UOINOdOld * 
Sjonpoid onenbe jo uoAWNSUOD 84} O} Pe}e|a1 
senssi ujjeay Jo eHpajmouy pue spseMmo} epnyye ,S|euoIssajosd aed YEOH * 
sealse 4SU-YDIY Ul PpjOuUSesu} 
Ajayes au} MO[aq SI UO!VEUIWeJUOD GOd JO |9A2| SSOUM USWIOM jueubad jo % ° 


JOAIY OOUAIME] 1S OU} WO Joyem de} Bulpsebes sapnyye JeuNnsuod 


Sys yyeay jenuajod ssasse 0} UO!}eZa}OeIeYO 

0} joalqns Jayem Buljuup ul s}UBUIWeUOS ;eo!LWAaYo AyWOUd Meu JO JOqUINN 
senssi uyeey jo eBHpajmouy pue EpNn}iye SJOWLUIMG 

JOAIY QOUBIME] “JS OU} Ul SWIMS JEU} UOHE|NdOd SpisseAl BY} JO UO!HOCOJq 
Buiwwims 40} ajqeyins ase pue Huyjsod 

ayenbepe aney jeu} ‘seyoeeg ueeso Buuenba.jy SIOWULUIMS JO uoIodold 
S]UBLUOINSEOW JO9jJa 0} OOIGns SPUNOAWOD jed|LWOYO JO JOQUINN 

O}1119} JIay} Ul SayOeaq ye SEONOU ayenbape sod OYM Saijl|/eAlO|UNW JO % 
Ajoyd19} 

sy} Ul Seyoeaq O1|Qnd ye Ayyenb 1ayem JOWUOW yey} SAI|EdIo|UNW JO % 
payenjene si Ayyenb sayem YoiyM ye o1qnd ay} Aq pasn seyoead 40 JOqUNN 


SHOLVOIGNI 


HL1VSH 


Ill @BSCUd O00E ATS - YOMEWIesY BOURLUIOLAd 


ev 


JOA GOUBIME] ‘JS BY} JO UONdEdJed Ul SeDUBYO Puke SESf) © 
JOAIY QOUSIME’] ‘JS BU} JO Bye}S BY} Jo eBHpa|mouy oqnd jo jeAe7 ° 
SanAoe 000Z ATS [12 JO UOKdeosed AyuNWWOS pue dI|GNd 


OK oO 


AWAnoe Jo Base Jed «@Ane}4 a7» Ul peysi|gnd sejoiye jo JEQqUINN °S 
ueid UOIEOIUNLWWLUOD 4184} Ul 
papnjou! pue paedojanep syonpoid jo Jaquuinu |2}0} Ou} O} paseduod jeus9e}U| OU} 
uo paysod aay!wWW09 Hulyeulps009 Jed sjonpojd SUOI}EOIUNLWULUOD jO JOQUINN “7 
sanIAoe SuONeolunWWOD Aq peyeioueb sajoiye |e Ul SSdUSJEJe4 JO JOqUINU 
ay} 0} paseduoo sebessow oiHayes}s pojebse} ay} 0} SEOUSJAjOI JO JOQUINN 49 
payeiauab sajoipe |je ul peuoljUeW oq Pinos Kau} 
SALLI} JO JOQUUNU OY} 0} paredUOd OOOZ ATS JO SOHIAIOe SUO!PES|UNWWOS OU} 
Aq peyeiouab sejoie ul pouojuew aie siouped jejusupedap SEU} JO JOQUINN “2 
POUO!USW SI }! SOLUI} JO JOQUUNU |2}O} OU} O} pasedwoo (seqeno 
“‘epeUed) SJUSWUUJ@AOCH OM} BU} YIIM payeloosse S! 0007 ATS SEWN JO JOQUINN |} 


e00z-200z| 200z-00z| 1002-0002] 0002-666! | 6661-8661 


SYOLVOIGNI 


SNOILVOINNNINOD 


Ill @BSCUd O00 ATS - YIOMALUBI4 BOUBWOLEd 


HSS om 5 Ve RINE Pat oats @ trian yh) niet hae 


a ee St + nd eee ~ ee ame ne ee eage—-Cee meee 


EHO AGINUMMC) 


ee ee ee — —— 


2A i ae) if ‘ 


Sham) cisermeneesh [96 a2 rw Me Asia Ses IP, A ao lore 
ganoliniaen athesiet te sede tice Set art patel 
(eo SDefater=5 colts i barctranr ae sinter’ 


apne Ty 782 reo BF sig OOO Ye Ys mene: ay 7 


. 7 5 
* . ‘ , z 
2 5 oh 
% . 
. * % 
3 = : : : 
° : ‘ ‘ 3 
= ' ‘ : : . 
: ers A * : aed : 
5 = : 
5 : : ; a 5 2 
« : ¥ 
- , oe 
S F as ; 
> 2 
‘ : i . 
F : ; 
* 2 


rar 
bey So 
a a 

Dee iar 

Vy TAS 
oe 


